Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug description.
std::shared_ptrio::IoStatistics ioStats passes it's naked pointer to FileSink::create(), then ioStats is released. Therefore FileSink holds an unknown pointer.
AddressSanitizer: heap-use-after-free on address 0x6130004f1998 at pc 0x7f6490a61210 bp 0x7f64aaed8e90 sp 0x7f64aaed8e88
[ERROR] WRITE of size 8 at 0x6130004f1998 thread T146 (Legacy Source T)
[ERROR] #0 0x7f6490a6120f in std::__atomic_base::fetch_add(unsigned long, std::memory_order) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/atomic_base.h:618
[ERROR] #1 0x7f6490a6120f in facebook::velox::io::IoStatistics::incRawBytesWritten(long) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/common/io/IoStatistics.cpp:58
[ERROR] #2 0x7f6492c0b0a3 in facebook::velox::dwio::common::FileSink::writeImpl(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&, std::function<unsigned long (facebook::velox::dwio::common::DataBuffer<char, void> const&)> const&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:82
[ERROR] #3 0x7f6492c0b907 in facebook::velox::dwio::common::LocalFileSink::write(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:168
[ERROR] #4 0x7f6492c0916e in facebook::velox::dwio::common::FileSink::writeWithLogging(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:60
[ERROR] #5 0x7f6492c0d76b in facebook::velox::dwio::common::FileSink::write(facebook::velox::dwio::common::DataBuffer<char, void>) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:53
[ERROR] #6 0x7f6495d13111 in facebook::velox::text::BufferedWriterSink::flush() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/text/writer/BufferedWriterSink.cpp:61 [ERROR] #7 0x7f6495d028de in facebook::velox::text::TextWriter::flush() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/text/writer/TextWriter.cpp:110
[ERROR] #8 0x7f64a2f4a39e in facebook::velox::connector::print::PrintSink::appendData(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/connectors/print/PrintSink.cpp:100
[ERROR] #9 0x7f64a39d729d in facebook::velox::exec::TableWriter::addInput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/exec/TableWriter.cpp:158
[ERROR] #10 0x7f6491bf0425 in facebook::velox::stateful::StatefulOperator::addInput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:38
[ERROR] #11 0x7f6491bf395e in facebook::velox::stateful::StatefulOperator::pushOutput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:76
[ERROR] #12 0x7f6491bf3ff5 in facebook::velox::stateful::StatefulOperator::getOutput() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:61
[ERROR] #13 0x7f6491bf3a89 in facebook::velox::stateful::StatefulOperator::pushOutput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:77
[ERROR] #14 0x7f6491bf3ff5 in facebook::velox::stateful::StatefulOperator::getOutput() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:61
[ERROR] #15 0x7f6491b8d6e9 in facebook::velox::stateful::StatefulTask::next(int&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulTask.cpp:116
[ERROR] #16 0x7f64a9cdf513 in velox4j::StatefulSerialTask::advance0(bool) /app/velox4j/src/main/cpp/main/velox4j/query/StatefulQueryExecutor.cc:136
[ERROR] #17 0x7f64a9cdfa0a in velox4j::StatefulSerialTask::advance() /app/velox4j/src/main/cpp/main/velox4j/query/StatefulQueryExecutor.cc:75
[ERROR] #18 0x7f64a9c96dca in upIteratorAdvance /app/velox4j/src/main/cpp/main/velox4j/jni/StaticJniWrapper.cc:77
[ERROR] #19 0x7f6ccccc5a6f ()
[ERROR] [ERROR] 0x6130004f1998 is located 24 bytes inside of 328-byte region [0x6130004f1980,0x6130004f1ac8)
[ERROR] freed by thread T143 (Source: MyTable) here:
[ERROR] #0 0x7f6ceb310d07 in operator delete(void*, unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:172
[ERROR] #1 0x7f64a2f4fc41 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >::deallocate(std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/ext/new_allocator.h:145
[ERROR] #2 0x7f64a2f4fc41 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::deallocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&, std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/alloc_traits.h:496
[ERROR] #3 0x7f64a2f4fc41 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::~__allocated_ptr() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/allocated_ptr.h:74
[ERROR] #4 0x7f64a2f4fc41 in std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:538 [ERROR]
[ERROR] previously allocated by thread T143 (Source: MyTable) here:
[ERROR] #0 0x7f6ceb30fe87 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:99
[ERROR] #1 0x7f64a2f4b5c7 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/ext/new_allocator.h:127
[ERROR] #2 0x7f64a2f4b5c7 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/alloc_traits.h:464
[ERROR] #3 0x7f64a2f4b5c7 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/allocated_ptr.h:98
[ERROR] #4 0x7f64a2f4b5c7 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics>(facebook::velox::io::IoStatistics*&, std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:648
[ERROR] #5 0x7f64a2f4b5c7 in std::__shared_ptr<facebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatorfacebook::velox::io::IoStatistics>(std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:1342
[ERROR] #6 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics::shared_ptr<std::allocatorfacebook::velox::io::IoStatistics>(std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:409
[ERROR] #7 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics std::allocate_shared<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics>(std::allocatorfacebook::velox::io::IoStatistics const&) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:863
[ERROR] #8 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics std::make_sharedfacebook::velox::io::IoStatistics() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:879 [ERROR] #9 0x7f64a2f4b5c7 in facebook::velox::connector::print::PrintSink::createWriter(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/connectors/print/PrintSink.cpp:44
[ERROR] [ERROR] Thread T146 (Legacy Source T) created by T143 (Source: MyTable) here:
[ERROR] #0 0x7f6ceb2b6676 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cpp:216
[ERROR] #1 0x7f6ce5cadb91 in os::create_thread(Thread*, os::ThreadType, unsigned long) (/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-2.el7_9.x86_64/lib/server/libjvm.so+0xc67b91)
[ERROR] #2 0x7f6ce59d49ac in JVM_StartThread (/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-2.el7_9.x86_64/lib/server/libjvm.so+0x98e9ac)